In any periodic musical note, there is always a fundamental frequency that determines the particular pitch of the note, as well as numerous harmonics which provide character or timbre to the musical note. It is the particular combination of the harmonic frequencies with the fundamental frequency that make, for example, a guitar and a violin playing the same note sound different from one another. The relationship of the amplitude of the fundamental frequency component to the amplitude of the harmonics created by an instrument is referred to as the spectral envelope. In a musical instrument such as a guitar, flute, or saxophone, the spectral envelope of a note played by the instrument expands and contracts more or less proportionally as the pitch of the note is shifted up or down.
Electronic pitch shifters are musical effects that receive an input note and produce an output note with a different pitch. Such effects are often used to allow a single musician to sound like several. For musical instruments, one can change the pitch of a note by sampling the sound from the instrument and playing back the sampled sounds at a rate that is either faster or slower than the rate at which the samples were recorded. The output notes created by this technique sound fairly natural because the spectral envelope of the pitch shifted sounds mimics how the spectral envelope of the sounds produced by the instrument vary with pitch.
In contrast to notes produced by musical instruments, the spectral envelope of vocal notes or sounds do not vary proportionately as the pitch of the vocal note varies. However, the relative magnitudes of the individual frequencies that make up this spectral envelope may change. Shifting the pitch of a vocal note by sampling a note as it is sung or spoken and playing the samples back at a different speed does not sound natural because the method varies the shape of the spectral envelope in proportion to the amount of pitch shift. In order to realistically shift the pitch of a vocal sound, a method is required for varying the frequency of the fundamental while only slightly varying the overall shape of the spectral envelope.
A device that shifts the pitch of vocal notes to create harmonies in real time is described in our prior U.S. Pat. No. 5,231,671 (the "'671 patent", the specification of which is herein incorporated by reference). The method of pitch shifting described in the '671 patent was adapted from an article, Lent, K. "An Efficient Method for Pitch Shifting Digitally Sampled Sounds," Computer Music Journal, Volume 13, No. 4, (1989) (also incorporated by reference herein, and hereafter referred to as the Lent method). The Lent method allows the pitch of a digitally sampled sound to be shifted without changing the spectral envelope. Briefly stated, the Lent method can be used to shift the pitch of a vocal note by replicating portions of a stored input signal at a rate that is faster or slower than the fundamental frequency input note. While this method of shifting the pitch of vocal notes works well, the pitch shifted notes do not sound completely natural, because the spectral envelope remains fixed as the pitches of the notes are varied.
As described above, there are two methods of electronically shifting the pitch of a note. The first method, referred to as resampling, or scaling in time the waveform modifies the spectral envelope in proportion to the amount of pitch shift. The Lent method more or less maintains the spectral envelope regardless of the amount of pitch shift. Neither of these two methods allow the spectral envelope to be varied in a controllable manner. Therefore, there is a need for a method of altering the spectral envelope of a musical note that is not dependent on the pitch of a note. With such a method, more realistic harmonies can be created. In addition, by changing the timbre of the note with or without changing the output pitch, it is possible to make one instrument sound like another, or one person's voice sound like another.